Image processor, image processing method, and computer-readable medium

ABSTRACT

There is provided an image processor for processing input image data having a luminance component and other components, which are defined in a prescribed color space. The image processor includes: a first emphasis processor configured to process first image data so as to emphasize the contrast of the first image data in accordance with a first emphasis amount, wherein the first image data correspond to the luminance component of the input image data; a second emphasis processor configured to process second image data so as to emphasize the chroma of the second image data in accordance with the first emphasis amount, wherein the second image data correspond to said other components of the input image data; and an image data generator configured to generate new image data by combining the contrast-emphasized first image data and the chroma-emphasized second image data.

BACKGROUND OF THE INVENTION

1. Field of the Invention

Embodiments described herein relate to a technique of performing imageprocessing on a dynamic-range-increased image, for example. 2.Description of the Related Art

In recent years, various kinds of image processing have come to beperformed on shot images to enhance their appearances when they areviewed. Among those kinds of image processing is processing disclosed inJP-A-2006-345509 which generates an image that are free of whiteout andblackout (such an image will be referred to as an HDR (high dynamicrange) image) through combining of images taken under different exposureconditions in proper regions.

However, the above combining processing is for increasing reproductionperformance relating to a shooting angle of view. Therefore, to producean image in which color space parameters are adjusted excessively butreality is not lost, it is necessary to newly perform drawing processingfrom the start using a shot image as a base image.

SUMMARY OF THE INVENTION

Exemplary embodiments provide an image processor that generates an imagein which color space parameters are adjusted using a shot image andreality is not lost.

According to one or more illustrative aspects of the present invention,there is provided an image processor for processing input image datahaving a luminance component and other components, which are defined ina prescribed color space. The image processor includes: a first emphasisprocessor configured to process first image data so as to emphasize thecontrast of the first image data in accordance with a first emphasisamount, wherein the first image data correspond to the luminancecomponent of the input image data; a second emphasis processorconfigured to process second image data so as to emphasize the chroma ofthe second image data in accordance with the first emphasis amount,wherein the second image data correspond to said other components of theinput image data; and an image data generator configured to generate newimage data by combining the contrast-emphasized first image data and thechroma-emphasized second image data.

According to one or more illustrative aspects of the present invention,there is provided an image processing method for processing input imagedata having a luminance component and other components, which aredefined in a prescribed color space. The method includes: (a) processingfirst image data so as to emphasize the contrast of the first image datain accordance with a first emphasis amount, wherein the first image datacorrespond to the luminance component of the input image data; (b)processing second image data so as to emphasize the chroma of the secondimage data in accordance with the first emphasis amount, wherein thesecond image data correspond to said other components of the input imagedata; and (c) generating new image data by combining thecontrast-emphasized first image data and the chroma-emphasized secondimage data.

According to one or more illustrative aspects of the present invention,there is provided a computer-readable medium storing a program forcausing the computer to process input image data having a luminancecomponent and other components, which are defined in a prescribed colorspace, in accordance with operations comprising: (a) processing firstimage data so as to emphasize the contrast of the first image data inaccordance with a first emphasis amount, wherein the first image datacorrespond to the luminance component of the input image data; (b)processing second image data so as to emphasize the chroma of the secondimage data in accordance with the first emphasis amount, wherein thesecond image data correspond to said other components of the input imagedata; and (c) generating new image data by combining thecontrast-emphasized first image data and the chroma-emphasized secondimage data.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of an imaging device according to anembodiment of the present invention;

FIG. 2 is a functional block diagram showing functions for performingimage alteration processing according to the invention which isperformed by an image processor;

FIG. 3 is a flowchart of an example image alteration process which isexecuted by the image processor of FIG. 2;

FIGS. 4A-4E show specific processing results of steps S1 and S2 whichare executed by a combiner;

FIG. 5 is a flowchart of an example contrast emphasis process which isexecuted by an image generator of the image processor of FIG. 2;

FIG. 6 is a flowchart of an example chroma emphasis process which isexecuted by the image generator of the image processor of FIG. 2;

FIGS. 7A-7D are example image data generated in executing the imagealteration process of FIG. 3;

FIG. 8 shows example contrast-emphasized image data generated by theimage alteration process of FIG. 3 using the data of FIGS. 7A-7D; and

FIG. 9 is a block diagram showing the hardware configuration of an imageprocessor according to another embodiment of the invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Exemplary embodiments of the present invention will be now describedwith reference to the drawings.

FIG. 1 is a block diagram of an imaging device. The imaging deviceincludes an image processor according to the embodiment of theinvention. This imaging device is equipped with an image capturing unit1, a drive controller 2, a CDS/ADC 3, a key input unit 4, a display unit5, an image recorder 6, a program memory 7, a RAM 8, a controller 9, andan image processor 10. The respective units are connected to each otherby a bus line.

The key input unit 4 includes a shutter key 41 for detecting a recordinginstruction of a photographer. The image processor 10 includes acombiner 11, and has functions of an image generator 12 and a noisereduction unit 13 (both described later).

The image capturing unit 1 incorporates an image sensor such as a CMOSsensor, RGB color filters placed on the image sensor, and a driver whichholds a light intensity distribution in the form of a distribution ofaccumulated charges for a prescribed time and outputting an analog imagesignal representing it to the CDS/ADC 3 under the control of the drivecontroller 2.

The image capturing unit 1 acquires plural image data (color image data)including underexposure image data, correct exposure image data, andoverexposure image data while changing an exposure condition (shutterspeed or aperture) by detecting a shooting instruction of a photographervia the shutter key 41, the controller 9, and the drive controller 2.

The CDS/ADC 3 is a circuit which receives analog image signalscorresponding to an optical image of a subject from the image capturingunit 1. The CDS/ADC 3 includes a CDS for holding received image signals,gain control amplifiers (AGC) for amplifying the image signals, and A/Dconverters (ADC) for converting amplified image signals into digitalimage signals.

A control relating to adjustment of the gain control amplifier isperformed according to an instruction from the drive controller 2.Therefore, even when plural images are acquired under the same exposurecondition, images corresponding to different conditions can be generatedby sequentially changing the characteristics of the RGB gain controlamplifiers or the image hue.

The key input unit 4 is equipped with, in addition to theabove-mentioned shutter key 41, various keys for detecting aninstruction about switching to a shooting mode for acquisition orrecording of an image relating to the invention, an instruction aboutdisplay switching, and other instructions. The display unit 5 isconfigured to display an image that has been subjected to combiningprocessing.

The image recorder 6 stores image data (an image file) produced bycombining processing according to the invention and subsequent JPEGcoding. The program memory 7 is stored with programs to be run by thecontroller 9 and the image processor 10. Programs are read by thecontroller 9 when necessary. The RAM 8 has a function of temporarilyholding data generated by each piece of processing and being processed.The controller 9 controls operations of the entire imaging device.

The image processor 10 is equipped with the combiner 11 which performscoding/decoding processing on image data and also performs the combiningprocessing according to the invention.

Next, functions relating to image alteration processing will bedescribed with reference to a functional block diagram of FIG. 2. Theimage alteration processing will be described below in detail isprocessing that processing subject images are subjected topixel-by-pixel addition combining and a resulting image is subjected tocontrast or chroma (saturation) emphasis.

In the combining processing, each of plural image data acquired throughthe image capturing unit 1 and the CDS/ADC 3 by consecutive shootingwhile an exposure condition (shutter speed, aperture, or gain controlvalue) is changed is separated into signals representing threecomponents defined in the YUV color space, that is, a luminance signalrepresenting a Y component and a blue color difference signal and a redcolor difference signal representing U and V components. The pluralluminance signals are subjected to pixel-by-pixel addition combining,and the set of plural blue color difference signals and the set ofplural red color difference signals are each subjected to pixel-by-pixeladdition combining.

Among a luminance signal (Y) and color difference signals (U and V) ofHDR combined image data generated by the above combining processing, theluminance signal is subjected to contrast emphasis processing and eachcolor difference signal is subjected to chroma (saturation) emphasisprocessing.

In particular, in the above processing, the entire image is notsubjected contrast or chroma emphasis uniformly. Instead, contrast orchroma emphasis is performed on a pixel-by-pixel basis using emphasisamounts that are set adaptively for the respective pixels.

As shown in FIG. 2, the image processor (image processor 10) is equippedwith the combiner 11, the image generator 12, and the noise reductionunit 13.

In the embodiment, as shown in FIG. 2, correct exposure image data of acorrect exposure value (hereinafter referred to as “correct exposureimage data”), image data of an exposure value that is larger than thecorrect exposure value (hereinafter referred to as “overexposure imagedata”), and image data of an exposure value that is smaller than thecorrect exposure value (hereinafter referred to as “underexposure imagedata”) are input to the combiner 11 as plural image data acquired underdifferent exposure conditions.

The term “correct exposure value” does not necessarily mean an exposurevalue that is suitable for shooting but means an exposure value that isan intermediate value between two exposure values that were used intaking an overexposure image and an underexposure image.

The combiner 11 generates HDR combined image data by performingpixel-by-pixel addition combining on the correct exposure image data,the overexposure image data, and the underexposure image data.

Although the kind of the combining processing performed by the combiner11 is not limited to particular processing, it is advantageous to employcombining processing that is stronger in the dynamic range increasingeffect than conventional combining processing for generating HDRcombined image data.

And method for making the dynamic range increasing effect stronger isnot limited to a particular method. Exemplary methods are increasing thedifference between an underexposure value and an overexposure value (arange variation) at the time of shooting, increasing the number ofimages to be combined, and changing a gain parameter of a hue or thelike.

In the embodiment, the combiner 11 executes the following series ofsteps as the combining processing.

In the embodiment, although not shown in any drawing, luminance imagedata having the correct exposure value and two color difference imagedata having the correct exposure value that have been separated from thecorrect exposure image data are input to the combiner 11 independently.Likewise, luminance image data having the overexposure value and twocolor difference image data having the overexposure value that have beenseparated from the overexposure image data are input to the combiner 11independently. And luminance image data having the underexposure valueand two color difference image data having the underexposure value thathave been separated from the underexposure image data are input to thecombiner 11 independently.

The combiner 11 generates combined image data by performingpixel-by-pixel addition combining on the luminance image data having thecorrect exposure value, the luminance image data having the overexposurevalue, and the luminance image data having the underexposure value. Thecombiner 11 also generates two combined image data by performingpixel-by-pixel addition combining on the two color difference image datahaving the correct exposure value, the two color difference image datahaving the overexposure value, and the two color difference image datahaving the underexposure value, respectively.

In the embodiment, in the luminance component combining processing, theluminance image data having the correct exposure value, the luminanceimage data having the overexposure value, and the luminance image datahaving the underexposure value are subjected to pixel-by-pixel additioncombining according to a prescribed combining ratio.

The term “combining ratio” means a mixing ratio of individual image datawhen two or more image data are subjected to pixel-by-pixel additioncombining. For example, if the combining ratio of the luminance imagedata having the correct exposure value, the luminance image data havingthe overexposure value, and the luminance image data having theunderexposure value is 70:30:0, combined luminance image data isobtained by mixing the luminance image data having the correct exposurevalue, the luminance image data having the overexposure value, and theluminance image data having the underexposure value at the ratio70:30:0.

On the other hand, in the color difference component combiningprocessing, the blue or red color difference image data having thecorrect exposure value, the blue or red color difference image datahaving the overexposure value, and the blue or red color differenceimage data having the underexposure value are subjected topixel-by-pixel addition combining according to a prescribed combiningratio which is the same as the prescribed combining ratio used for theluminance component combining processing.

As a result, the combiner 11 outputs a set of combined luminance imagedata generated by the luminance component combining processing and twocombined color difference component image data generated by the colordifference component combining processing.

The image generator 12 performs contrast emphasis processing on thecombined luminance image data that is output from the combiner 11 andperforms chroma emphasis processing on each of the two combined colordifference component image data that are output from the combiner 11.The image generator 12 is equipped with a contrast emphasis processor 31which performs the contrast emphasis processing and a chroma emphasisprocessor 32 which performs the chroma emphasis processing.

The contrast emphasis processor 31 emphasizes contrast locally byperforming unsharpening mask processing in two frequency bands (e.g., alow-frequency band and a medium-frequency band) on the combinedluminance image data (the combined luminance image data is smoothed). Noparticular limitations are imposed on the filter that is used for suchsmoothing. Although an ordinary LPF (lowpass filter) may be employed, itis advantageous to use a nonlinear filter such as an edge-maintaining sfilter or a bilateral filter.

The contrast emphasis processor 31 is equipped with a combined luminanceimage acquiring unit 41, ε filters 42A and 42B, contrast componentgenerators 43A and 43B, a contrast component combiner 44, and a contrastemphasizing unit 45.

The functions of the individual components of the contrast emphasisprocessor 31 will be described as appropriate in describing a contrastemphasis process later with reference to a flowchart of FIG. 3.

The chroma emphasis processor 32 performs processing of chroma on eachcombined color difference image data using a prescribed gain(hereinafter referred to as a UV gain) as an emphasis amount.

The chroma emphasis processing is generally divided into two kinds ofprocessing, that is, total emphasis processing and partial emphasisprocessing. The total emphasis processing is processing of emphasizingchroma so that the entire image looks showy. The partial emphasisprocessing is processing of emphasizing chroma using emphasis amountsthat are linked with contrast component emphasis amounts that have beenused in the contrast emphasis processing.

In the following, the emphasis amounts for the contrast emphasisprocessing will be referred to as contrast component emphasis amountsand the emphasis amounts for the chroma emphasis processing will bereferred to as chroma component emphasis amounts. Furthermore, thechroma component emphasis amounts are generally divided into an emphasisamount used for the total emphasis processing and emphasis amounts usedfor the partial emphasis processing. The former will be referred to as atotal chroma component emphasis amount and the latter will be referredto as partial chroma component emphasis amounts.

A series of steps executed by the chroma emphasis processor 32 will bereferred to as a chroma emphasis process. The chroma emphasis processor32, which executes the chroma emphasis process, includes a combinedcolor difference image acquiring unit 51, a total chroma emphasizingunit 52, a contrast component emphasis amounts acquiring unit 53, and apartial chroma emphasizing unit 54.

The image generator 12 is also equipped with an emphasized imageacquiring unit 33. The emphasized image acquiring unit 33 acquiresemphasized image data which is a combination of combined luminance imagedata that has been generated by the contrast emphasis processor 31through the contrast emphasis processing and two combined colordifference image data that have been generated by the chroma emphasisprocessor 32 through the chroma emphasis processing.

The emphasized image data acquired by the emphasized image acquiringunit 33 is provided to the noise reduction unit 13. The noise reductionunit 13 performs noise reduction processing on the emphasized imagedata.

Next, an image alteration process which is executed by the imageprocessor of FIG. 2 having the above functional configuration will bedescribed with reference to a flowchart of FIG. 3. FIG. 3 is a flowchartof an example image alteration process.

At step S1, the combiner 11 acquires plural image data which have beenproduced by consecutive shooting or the like at different exposurevalues. At step S2, the combiner 11 generates HDR combined image data byperforming pixel-by-pixel addition combining on the plural image datahaving the different exposure values.

More specifically, in the embodiment, HDR combined image data isgenerated in such a manner that correct exposure image data,overexposure image data, and underexposure image data are acquired atstep S1 and subjected to pixel-by-pixel addition combining at step S2.

FIGS. 4A-4E show specific processing results of steps S1 and S2 whichare executed by the combiner 11. FIG. 4A shows example underexposureluminance image data acquired at step S1. FIG. 4B shows example correctexposure luminance image data acquired at step S1. FIG. 4C shows exampleoverexposure luminance image data acquired at step S1.

FIG. 4D shows example combined image data generated at step S2 throughpixel-by-pixel addition combining. FIG. 4E shows example combined imagedata generated at step 52 in which the effect of combining is strongerthan in the example combined image data of FIG. 4D. Increasing theeffect of combining makes more remarkable an advantage that an HDRcombined image is obtained in which color space parameters are adjustedexcessively but reality is not lost.

Therefore, in the embodiment, between the combined image data of FIGS.4D and 4E, the combined image data of FIG. 4E is provided to the imagegenerator 12 as a processing subject and subjected there to steps S3-S5shown in FIG. 3.

At step S3, the contrast emphasis processor 31 of the image generator 12performs contrast emphasis processing on combined luminance image dataof the combined image data which was generated at step S2.

At step S4, the chroma emphasis processor 32 of the image generator 12performs chroma emphasis processing on each of two combined colordifference image data of the combined image data which was generated atstep S2.

The details of step S3 (contrast emphasis process) will be describedlater with reference to a flowchart of FIG. 5, and the details of stepS4 (chroma emphasis process) will be described later with reference to aflowchart of FIG. 6.

At step S5, the emphasized image acquiring unit 33 acquires emphasizedimage data which is a combination of combined luminance image data thatwas generated at step S3 through the contrast emphasis processing andtwo combined color difference image data that were generated at step S4through the chroma emphasis processing.

At step S6, the noise reduction unit 13 performs noise reductionprocessing on the emphasized image data acquired at step S5.Noise-reduced emphasized image data is output from the noise reductionunit 13, whereupon the image alteration process is finished.

Next, the contrast emphasis process and the chroma emphasis processwhich are steps S3 and S4 of the above-described image alterationprocess, respectively, will be described in detail individually in thisorder.

First, the contrast emphasis process which is executed at step S3 willbe described in detail with reference to a flowchart of FIG. 5. FIG. 5is a flowchart of an example contrast emphasis process.

At step S21, the combined luminance image acquiring unit 41 of thecontrast emphasis processor 31 shown in FIG. 2 acquires combinedluminance image data of combined image data that is output from thecombiner 11.

At step S22, the contrast emphasis processor 31 sets, as a pixel ofinterest (subject of processing), a pixel that is located at aprescribed position (coordinates) among the pixels constituting thecombined luminance image data. When data (a pixel value) of the pixel ofinterest of the combined luminance image data has been provided to eachof the ε filters 42A and 42B and the contrast component generators 43Aand 43B, the process moves to step S23.

At step S23, the ε filters 42A and 42B and the contrast componentgenerators 43A and 43B generate two contrast components of the pixel ofinterest from the data (pixel value) of the pixel of interest of thecombined luminance image data using ε filters of two frequency bands (amedium-frequency band and a low-frequency band).

In the embodiment, a component for one of two kinds of contrast forwhich components are to be generated is generated using the ε filter formedium-frequency band emphasis which has a small filter size. Therefore,this kind of contrast will be referred to as medium-frequency bandcontrast. A component for the other kind of contrast is generated usingthe c filter for low-frequency band emphasis which has a large filtersize. This kind of contrast will be referred to as low-frequency bandcontrast.

In the embodiment, a medium-frequency band contrast component isgenerated by the ε filter 42A and the contrast component generator 43Aby executing the following series of steps.

The ε filter 42A applies the ε filter for medium-frequency band emphasisto the data of the pixel of interest of the combined luminance imagedata and provides resulting data (hereinafter referred to as filtereddata) to the contrast component generator 43A.

The contrast component generator 43A generates a medium-frequency bandcontrast component of the pixel of interest by taking the differencebetween the data of the pixel of interest of the combined luminanceimage data and the filtered data of the pixel of interest.

More specifically, a medium-frequency band contrast component of thepixel of interest is generated according to the following Equation (1):

Y_contrast_(—)1=Y _(—) HDR−Y_ε_(—)1   (1)

where Y_contrast_1 is the medium-frequency band contrast component ofthe pixel of interest, Y_HDR is the data (pixel value) of the pixel ofinterest of the HDR combined luminance image, and Y_ε_1 is the filtereddata of the pixel of interest generated using the medium-frequency bandε filter.

On the other hand, in the embodiment, a low-frequency band contrastcomponent is generated by the ε filter 42B and the contrast componentgenerator 43B by executing the following series of steps.

The ε filter 42B applies the ε filter for low-frequency band emphasis tothe data of the pixel of interest of the combined luminance image dataand supplies resulting data (hereinafter referred to as filtered data)to the contrast component generator 43B.

The contrast component generator 43B generates a low-frequency bandcontrast component of the pixel of interest by taking the differencebetween the data of the pixel of interest of the combined luminanceimage data and the filtered data of the pixel of interest.

More specifically, a low-frequency band contrast component of the pixelof interest is generated according to the following Equation (2):

Y_contrast_(—)2=Y _(—) HDR−Y_ε_(—)2   (2)

where Y_contrast_2 is the low-frequency band contrast component of thepixel of interest, Y_HDR is the data (pixel value) of the pixel ofinterest of the HDR combined luminance image, and Y_ε_2 is the filtereddata of the pixel of interest generated using the low-frequency band εfilter.

The thus-generated two contrast components of the pixel of interest,that is, the medium-frequency band contrast component of the pixel ofinterest generated by the contrast component generator 43A and thelow-frequency band contrast component of the pixel of interest generatedby the contrast component generator 43B, are supplied to the contrastcombiner 44. Then, the process moves to step S24 shown in FIG. 5.

At step S24, the contrast combiner 44 generates a combined contrastcomponent of the pixel of interest by adding together the two contrastcomponents of the pixel of interest, that is, the medium-frequency bandcontrast component of the pixel of interest and the low-frequency bandcontrast component of the pixel of interest, according to the followingEquation (3):

Y_contrast=Y_contrast_(—)1+Y_contrast_(—)2   (3)

where Y_contrast is the combined contrast component of the pixel ofinterest, Y_contrast_1 is the medium-frequency band contrast componentof the pixel of interest calculated according to Equation (1) andY_contrast_2 is the low-frequency band contrast component of the pixelof interest calculated according to Equation (2).

The combined contrast component of the pixel of interest generated bythe contrast combiner 44 is provided to the contrast emphasizing unit45. Then, the process moves to step S25 shown in FIG. 5.

At step S25, the contrast emphasizing unit 45 adjusts and limits thecombined contrast component of the pixel of interest.

More specifically, in the embodiment, the combined contrast component ofthe pixel of interest is adjusted according to the following Equation(4):

Y_contrast=Y_contrast×(Y_gain_lev/100)×(Y_adj_lev/100)   (4)

In Equation (4), Y_contrast on the left side is the adjusted combinedcontrast component of the pixel of interest and Y_contrast on the rightside is the contrast component of the pixel of interest before theadjustment. The parameter Y_gain_lev is the level (%) of a gain that isset uniformly for the entire combined luminance image data (hereinafterreferred to as a luminance component gain). The parameter Y_adj_lev isthe level (%) of an adjustment amount (hereinafter referred to as aluminance component adjustment amount) that is set for the data of theindividual pixel of interest of the combined luminance image data.

Luminance component adjustment amounts Y_adj_lev are set individuallyfor the respective pixels of the combined luminance image according torespective pixel data Y_HDR. Therefore, the luminance componentadjustment amount Y_adj_lev of the pixel of interest is variedadaptively according to the pixel data Y_HDR, whereby the combinedcontrast component Y_contrast is adjusted adaptively on a pixel-by-pixelbasis. This makes it possible to reduce adverse effects, such asblackout, whiteout, and color saturation, of pixel value saturation dueto contrast emphasis.

In the embodiment, the combined contrast component of the pixel ofinterest is limited according to the following Equation (5) if it islarger than 0 (Y_contrast>0) and according to the following Equation (6)if not:

Y_contrast=fmin(Y_contrast, Y_contrast_max_lev)   (5)

Y_contrast=fmax(Y_contrast, |Y_contrast_max_lev|)   (6)

In Equations (5) and (6), Y_contrast on the left side is the limitedcombined contrast component of the pixel of interest and Y_contrast onthe right side is the combined contrast component of the pixel ofinterest before the limitation. The parameter Y_contrast_max_lev is apreset upper limit level of the combined contrast component. In Equation(5), fmin(α, β) is a function which outputs the minimum value of α andβ. In Equation (6), fmax(α, β) is a function which outputs the maximumvalue of α and β.

Then, at step S26 shown in FIG. 5, the contrast emphasizing unit 45 addsa combined contrast component of the pixel of interest as adjusted andlimited at step S25 to the data of the pixel of interest of the combinedluminance image data.

As a result, the data of the pixel of interest of the combined luminanceimage data is contrast-emphasized by the combined contrast component ofthe pixel of interest as adjusted and limited at step S25.

More specifically, the data of the pixel of interest of the combinedluminance image data is contrast-emphasized according to the followingEquation (7):

Y_result=Y_HDR+Y_contrast

In Equation (7), Y_result is the data (pixel value) of the pixel ofinterest that has been subjected to the contrast emphasis process. Theparameter Y_HDR is the data (pixel value) of the pixel of interestbefore being subjected to the contrast emphasis process. The parameter Ycontrast is the combined contrast component of the pixel of interestthat has been adjusted according to Equation (4) and limited accordingto Equation (5) or (6), and is also a contrast component emphasisamount.

At step S27, the contrast emphasis processor 31 judges whether or notall the pixels constituting the combined luminance image have been setas a pixel of interest. If there remains a pixel(s) that has not beenset as a pixel of interest yet (S27: no), the process returns to stepS22.

That is, the loop of steps S22-S27 is executed and the data of the pixelof interest is contrast-emphasized (updated) every time one of thepixels constituting the combined luminance image is set as a subject ofattention. As described above, the degree of contrast emphasis is variedadaptively according to the value of the data Y_HDR of the pixel ofinterest that is not subjected to the contrast emphasis.

If the last pixel is set as a subject of attention at step S22 and thensteps S23-S26 are executed, the judgment result of step S27 turns “yes”because all the pixels constituting the combined luminance image havebeen set as a pixel of interest (and their pixel values have beenupdated to values Y_result (see Equation (7)). Then, the contrastemphasis process is finished.

That is, the contrast emphasis process which is step S3 of the imagealteration process of FIG. 3 is thus completed. The chroma emphasisprocess which is step S4 of the image alteration process of FIG. 3 isthen executed.

The chroma emphasis process (step S4) will be described below in detailwith reference to a flowchart of FIG. 6. FIG. 6 is an example chromaemphasis process.

At step S41, the combined color difference image acquiring unit 51 ofthe chroma emphasis processor 32 shown in FIG. 2 acquires two combinedcolor difference image data of the combined image data that is outputfrom the combiner 11.

At step S42, the chroma emphasis processor 32 sets, as a pixel ofinterest, a pixel that is located at a prescribed position (coordinates)among the pixels constituting each combined color difference image data

Although in the embodiment the chroma emphasis processor 32 operatesasynchronously with the contrast emphasis processor 31, the followingdescription will be made with an assumption that the pixel of interestof each combined color difference image data corresponds to (i.e., islocated at the same position (coordinates) as) that of the combinedluminance image data.

At step S43, the total chroma emphasizing unit 52 performs total chromaemphasis processing on the data of the pixel of interest of eachcombined color difference image data using a total chroma componentemphasis amount.

More specifically, in the embodiment, total chroma emphasis processingis performed on the data of the pixel of interest of each combined colordifference image data according to the following Equations (8) and (9).

U _(—) HDR′=U _(—) HDR×UV_gain_lev_al/100   (8)

V _(—) HDR′=V _(—) HDR×UV_gain_lev_al/100   (9)

In Equation (8), U_HDR′ is the color difference value (U value) of thepixel of interest, as subjected to the total chroma emphasis processing,of the combined color difference image data. The parameter U_HDR is thecolor difference value (U value) of the pixel of interest, before beingsubjected to the total chroma emphasis processing, of the combined colordifference image data.

In Equation (9), V_HDR′ is the color difference value (V value) of thepixel of interest, as subjected to the total chroma emphasis processing,of the combined color difference image data. The parameter V_HDR is thecolor difference value (V value) of the pixel of interest, before beingsubjected to the total chroma emphasis processing, of the combined colordifference image data.

In Equations (8) and (9), UV_gain_lev_al is a total gain (%) as thetotal chroma component emphasis amount that is set uniformly for eachentire combined color difference image data.

At step S44, the contrast component emphasis amount acquiring unit 53acquires a contrast component emphasis amount that has been used for thedata of the pixel of interest of the combined luminance image data inthe contrast emphasis process.

More specifically, a contrast component emphasis amount is acquiredaccording to the following Equation (10).

Y_emphasis_lev=|Y_contrast|  (10)

In Equation (10), Y_emphasis_lev is the acquired contrast componentemphasis amount. The parameter Y_contrast is the combined contrastcomponent of the pixel of interest that has been adjusted according toEquation (4) and limited according to Equation (5) or (6) and that hasbeen added to the data of the pixel of interest (pixel value Y_HDR) thatis not subjected to the contrast emphasis process (see Equation (7)).

At step S45, the partial chroma emphasizing unit 54 performs partialchroma emphasis processing on the data of the pixel of interest, assubjected to the total chroma emphasis processing at step S43, of eachcombined color difference image data using a partial chroma componentemphasis amount which linked with the contrast component emphasis amountacquired at step S44.

More specifically, in the embodiment, partial chroma emphasis processingis performed on the U value of the pixel of interest, as subjected tothe total chroma emphasis processing, of the combined color differenceimage data according to the following Equation (11):

$\begin{matrix}{{U\_ result} = {{U\_ HDR}^{\prime} + {{U\_ HDR} \times {\left( {{UV\_ gain}{\_ lev}{\_ part} \times {Y\_ emphasis}{{\_ lev}/{Y\_ contrast}}{\_ max}{\_ lev}} \right)/100} \times {UV\_ adj}{{\_ lev}/100}}}} & (11)\end{matrix}$

In Equation (11), U_result is the U value of the pixel of interest,subjected to the total chroma emphasis processing and the partial chromaemphasis processing, of the combined color difference image data. Theparameter U_HDR′ is the U value of the pixel of interest, as subjectedto the total chroma emphasis processing according to Equation (8), ofthe combined color difference image data. The parameter UV_gain_lev_partis a partial gain (%). The parameter Y_emphasis_lev is the contrastcomponent emphasis amount acquired according to Equation (10). Theparameter Y_contrast_max_lev is an upper limit level that is preset as acombined contrast component.

The parameter (UV_gain_lev_part×Y_emphasis_lev/Y_contrast_max_lev) is apartial chroma emphasis amount (%) which is linked with the contrastcomponent emphasis amount. The parameter UV_adj_lev is the level (%) ofan adjustment amount (hereinafter referred to as a color differenceadjustment amount) that is set for the data of the individual pixel ofinterest of the combined color difference image data.

Data UV_HDR is the larger one of the absolute values of the U valueU_HDR and the V value V_HDR of the pixel of interest of the combinedcolor difference image data (see the following Equation (12)):

UV _(—) HDR=fmax(|U _(—) HDR|, |V _(—) HDR)   (12)

where fmax(α, β) is a function which outputs the maximum value of α andβ.

Color difference adjustment amount UV_adj_lev are set individually forthe respective pixels of the combined color difference image accordingto respective data UV_HDR. Therefore, the color difference adjustmentamount UV_adj_lev of the pixel of interest is varied adaptivelyaccording to the data UV_HDR.

As a result, the degree of partial chroma emphasis on the U value U_HDRof the pixel of interest is adjusted adaptively on a pixel-by-pixelbasis. This makes it possible to reduce adverse effects, such asblackout, whiteout, and color saturation, of pixel value saturation dueto chroma emphasis.

In the embodiment, partial chroma emphasis processing is performed onthe V value of the pixel of interest, as subjected to the total chromaemphasis processing, of the combined color difference image dataaccording to the following Equation (13):

$\begin{matrix}{{V\_ result} = {{V\_ HDR}^{\prime} + {{V\_ HDR} \times {\left( {{UV\_ gain}{\_ lev}{\_ part} \times {Y\_ emphasis}{{\_ lev}/{Y\_ contrast}}{\_ max}{\_ lev}} \right)/100} \times {UV\_ adj}{{\_ lev}/100}}}} & (13)\end{matrix}$

In Equation (13), V_result is the V value of the pixel of interest,subjected to the total chroma emphasis processing and the partial chromaemphasis processing, of the combined color difference image data. Theparameter V_HDR′ is the V value of the pixel of interest, as subjectedto the total chroma emphasis processing according to Equation (9), ofthe combined color difference image data.

Like Equation (11), Equation (13) includes(UV_gain_lev_part×Y_emphasis_lev/Y_contrast_max_lev) as the partialchroma emphasis amount (%) which is linked with the contrast componentemphasis amount, as well as the color difference adjustment amountUV_adj_lev.

At step S46 shown in FIG. 6, the chroma emphasis processor 32 judgeswhether or not all the pixels constituting each combined colordifference image have been set as a pixel of interest. If there remainsa pixel(s) that has not been set as a pixel of interest yet (S46: no),the process returns to step S42.

That is, the loop of steps S42-S46 is executed and the data of the pixelof interest is chroma-emphasized (updated) every time one of the pixelsconstituting each combined color difference image is set as a subject ofattention. As described above, the degree of chroma emphasis is variedin link with the degree of contrast emphasis and adaptively according tothe values of the data U_HDR and V_HDR of the pixel of interest that isnot subjected to the chroma emphasis.

If the last pixel is set as a subject of attention at step S42 and thensteps S43-S45 are executed, the judgment result of step S46 turns “yes”because all the pixels constituting each combined luminance image havebeen set as a pixel of interest (and their pixel values have beenupdated to values U_result (see Equation (11) or values V_result (seeEquation (13)). Then, the chroma emphasis process is finished.

The chroma emphasis process which is step S4 of the image alterationprocess of FIG. 3 is thus completed, and the process of FIG. 3 moves tostep S5. At step S5, a combination of the combined luminance image datathat was contrast-emphasized at step S3 and the two combined colordifference image data that were chroma-emphasized at step S4 is acquiredby the emphasized image acquiring unit 33 as emphasized image data.

As described above, where combined image data is separated into combinedluminance image data and two combined color difference image data, thecontrast emphasis processor 31 shown in FIG. 2 emphasizes the contrastof the combined luminance image data using prescribed contrast componentemphasis amounts.

On the other hand, the chroma emphasis processor 32 shown in FIG. 2emphasizes the chroma of each combined color difference image data usingchroma component emphasis amounts which vary in link with the respectivecontrast component emphasis amounts.

This makes it possible to provide an advantage that an HDR combinedimage is obtained in which color space parameters are adjustedexcessively but reality is not lost.

In particular, this advantage is made more remarkable by the featuresthat the contrast emphasis processor 31 performs contrast emphasisprocessing on data of a pixel located at each position (set ofcoordinates) in a combined luminance image using a contrast componentemphasis amount which varies depending on the position and the chromaemphasis processor 32 performs chroma emphasis processing on data of apixel, located at the same position (coordinates) as the above pixel inthe combined luminance image, in each combined color difference imageusing a contrast component emphasis amount which varies in link with thecontrast component emphasis amount for the above pixel in the combinedluminance image.

This advantage is clearly seen from FIGS. 7A-7D and FIG. 8. FIGS. 7A-7Dshow example image data that were generated in executing the imagealteration process of FIG. 3, and correspond to the example image dataof FIG. 4E.

FIG. 7A shows example combined image data that was generated by step S2.

FIG. 7B shows example filtered data of the entire image that wasgenerated by applying an ε filter (size: 33×33 pixels) formedium-frequency band contrast emphasis to the entire combined imagedata of FIG. 7A at step S23 shown in FIG. 5 of the image alterationprocess of FIG. 3.

FIG. 7C shows example filtered data of the entire image that wasgenerated by applying an s filter (size: 129×129 pixels) forlow-frequency band contrast emphasis to the entire combined image dataof FIG. 7A at step S23 shown in FIG. 5 of the image alteration processof FIG. 3.

FIG. 7D shows an example contrast component of the entire image that wasgenerated using the data of FIGS. 7A-7C at steps S24 and S25 shown inFIG. 5 of the image alteration process of FIG. 3.

FIG. 8 shows example contrast-emphasized image data that was generatedby the image alteration process of FIG. 3 using the data of FIGS. 7A-7D.It is seen that this contrast-emphasized image is such that color spaceparameters are adjusted excessively but reality is not lost.

The invention is not limited to the above embodiment but encompassesmodifications, improvements, etc. made in such a range that the objectof the invention can be attained.

For example, in the above embodiment, the ε filters are used assmoothing filters in the contrast emphasis process which is executed bythe contrast emphasis processor 31. However, as mentioned above, thefilter used for that purpose is not limited to an ε filter. For example,an ordinary LPF may be employed when it is desired to generate an imagewhich includes an intentional undershoot or an overshoot which isgenerated in making local contrast emphasis.

Although the above embodiment employs two frequency bands (alow-frequency band and a high-frequency band) in the contrast emphasisprocess, the invention is not limited to such a case and may employthree or more frequency bands. For example, a high-frequency band may beemployed in addition to the low-frequency and high-frequency bands, inwhich case sharpness can be increased by emphasizing high-frequencycomponents.

In the above embodiment, no consideration is given to a case that aprocessing subject image of partial chroma emphasis processing includesa face image region. However, it is possible to detect a face imageregion using a known face image region detecting technique and performpartial chroma emphasis processing with small emphasis amounts takingthe detected face image region into consideration. This processing cansolve a problem that a skin-color hue texture or the like of a face arelost due to excessive chroma emphasis.

More specifically, a contrast component emphasis amount and a chromacomponent emphasis amount to be applied to luminance image data andcolor difference image data, respectively, of a face image region whenit is detected are set in advance. These emphasis amounts may be smallerthan (about 80% of) contrast component emphasis amounts and chromacomponent emphasis amounts described above, respectively.

If a face image region is detected, an image generated through partialemphasis processing with contrast component emphasis amounts and chromacomponent emphasis amounts and an image generated through partialemphasis processing with the contrast component emphasis amount and thechroma component emphasis amount for a face image region are generatedat the above-described step S3 and S4. A combined image is obtained byα-blending these images on a pixel-by-pixel basis (α-blending: blendingwith transparency adjustment).

The α-blending may be performed in such a manner that the proportion ofthe image generated through the partial emphasis processing with thecontrast component emphasis amount and the chroma component emphasisamount for a face image region is set higher in the detected face imageregion and its vicinity and the proportion of the image generated atsteps S3 and S4 through the partial emphasis processing with thecontrast component emphasis amounts and the chroma component emphasisamounts is set higher in the other region.

In the above embodiment, three image data having different exposurevalues are combined together by pixel-by-pixel addition combining, thenumber of image data to be combined together is not limited to three.That is, the invention can broadly applied to a case that an arbitrarynumber of (two or more) image data are subjected to pixel-by-pixeladdition combining.

Various image data as subjects of the image alteration process are notlimited to example image data described in the above embodiment. Forexample, exposure values that are set in generating plural image data tobe combined together are not limited to example values described in theembodiment and may be arbitrary values as long as they are differentfrom each other.

Although not described in the above embodiment, it is advantageous thatpositional deviations between plural image data having differentexposure values be corrected in advance because they will be subjectedto pixel-by-pixel addition combining.

Although the image processing according to the embodiment is a digitalcamera, the invention is not limited to such a case but applied togeneral electronic devices having an image processing function. Thus,the invention can broadly be applied to digital photo frames, personalcomputers, portable navigation system, portable game machines, etc.

The above-described processes can be executed by either hardware orsoftware. FIG. 9 is a block diagram showing the hardware configurationof an image processor according to another embodiment of the inventionin which the above-described processes are executed by software.

The image processor of FIG. 9 includes a CPU (central processing unit)201, a ROM (read-only memory) 202, a RAM (random access memory) 203, abus 204, an input/output interface 205, an input unit 206, an outputunit 207, a storage unit 208, a communication unit 209, and a drive 210.

The CPU 201 executes various processes according to programs recorded inthe ROM 202. Or the CPU 201 executes various processes according toprograms that have been loaded into the RAM 203 from the storage unit208.

Data that are necessary when the CPU 201 executes various processes arealso stored in the RAM 203 as appropriate.

For example, in this embodiment, programs for realizing the combiner 11,the image generator 12, and the noise reduction unit 13 are stored inthe ROM 202 or the storage unit 208. Therefore, the functions of thecombiner 11, the image generator 12, and the noise reduction unit 13 canbe realized when the CPU 201 runs those programs.

The CPU 201, the ROM 202, and the RAM 203 are connected to each other bythe bus 204. The bus 204 is also connected to the input/output interface205. The input unit 206, the output unit 207, the storage unit 208, andthe communication unit 209 are connected to the input/output interface205.

The input unit 206 is a unit having various buttons etc., and receivesuser instructions and various kinds of information.

The output unit 207 outputs various kinds of information. For example,the output unit 207 includes a display unit (not shown) and displays acomposed image that is represented by output data of the combiner 11.

The storage unit 208 is a hard disk drive, a DRAM (dynamic random accessmemory), or the like, and stores various kinds of data.

The communication unit 209 controls a communication that is performedwith another apparatus (not shown) over a network such as the Internet.

When necessary, the drive 210 is also connected to the input/outputinterface 205 and a removable medium 211 such as a magnetic disk, anoptical disc, a magneto-optical disc, or a semiconductor memory isinserted into it. A program that is read from the removable medium 211by the drive 210 is installed in the storage unit 208 if necessary. Likethe storage unit 208, the removable medium 211 can store various datasuch as image data.

To execute a certain process by software, a program as the software isinstalled in a computer or the like from a network or a recordingmedium. The computer may be incorporated in dedicated hardware.Alternatively, the computer may be a computer such as a general-purposepersonal computer which can perform various functions when variousprograms are installed therein.

A recording medium containing such a program is the removable medium 211that is distributed separately from the device main body to provide theuser with the program, a recording medium that is provided to the userin such a state as to be incorporated in the device main body, or a likerecording medium. The removable medium 211 is a magnetic disk (includinga floppy disk), an optical disc, a magneto-optical disc, or the like.Example optical discs are a CD-ROM (compact disc-read only memory) are aDVD (digital versatile disc). An example magneto-optical disc is an MD(mini disc). An example recording medium that is provided to the user insuch a state as to be incorporated in the device main body is the ROM202 which is stored with programs and the hard disk drive of the storageunit 208.

Steps of a program recorded in a recording medium that are described inthis specification may naturally be executed in the time-series order asdescribed. However, they need not always be executed in time-seriesorder, that is, they may be executed parallel or individually.

1. An image processor for processing input image data having a luminancecomponent and other components, which are defined in a prescribed colorspace, the image processor comprising: a first emphasis processorconfigured to process first image data so as to emphasize the contrastof the first image data in accordance with a first emphasis amount,wherein the first image data correspond to the luminance component ofthe input image data; a second emphasis processor configured to processsecond image data so as to emphasize the chroma of the second image datain accordance with the first emphasis amount, wherein the second imagedata correspond to said other components of the input image data; and animage data generator configured to generate new image data by combiningthe contrast-emphasized first image data and the chroma-emphasizedsecond image data
 2. The image processor according to claim 1, whereinthe first emphasis processor is configured to process the first imagedata so as to emphasize the contrast of respective pixels of the firstimage data in accordance with the first emphasis amount, wherein thefirst emphasis amount is varied depending on the respective pixels whichare located at different positions of the first image data, and whereinthe second emphasis processor is configured to process the second imagedata so as to emphasize the chroma of respective pixels of the secondimage data in accordance with a second emphasis amount, wherein thesecond emphasis amount is varied depending on the respective pixelswhich are located at different positions of the second image data, andthe second emphasis amount is varied depending on variation of the firstemphasis amount.
 3. The image processor according to claim 2, whereinthe second emphasis processor comprises: a total chroma emphasisprocessor configured to emphasize the chroma of the respective pixels ofthe second image data in accordance with a third emphasis amount,wherein the third emphasis amount is uniform for the respective pixelswhich are located at different positions of the second image data; andan acquiring unit configured to acquire the second emphasis amount, andwherein the second emphasis processor is configured to emphasize thechroma of the second image data by emphasizing the chroma of therespective pixels of the second image data in accordance with the thirdemphasis amount and emphasizing the chroma of the respective pixels ofthe second image data in accordance with the second emphasis amountacquired by the acquiring unit
 4. The image processor according to claim1, further comprising: an image region detector configured to detect aface image region of an object from the input image data, wherein: ifthe image region detector detects the face image region of the object,the first emphasis processor emphasizes the contrast of the first imagedata in accordance with a fourth emphasis amount that is smaller thanthe first emphasis amount, and the second emphasis processor emphasizesthe chroma of the second image data in accordance with a fifth emphasisamount that is varied depending on variation of the fourth emphasisamount.
 5. The image processor according to claim 4, wherein if theimage region detector detects the face image region of the object fromthe input image data, the first emphasis processor emphasizes thecontrast of a region around the face image region in the first imagedata.
 6. The image processor according to claim 4, wherein if the imageregion detector detects the face image region of the object from theinput image data, the second emphasis processor emphasizes the chroma ofa region around the face image region in the second image data.
 7. Theimage processor according to claim 1, wherein the input image data areimage data obtained by combining a plurality of images captured indifferent exposure conditions, and dynamic-range of the input image datais increased.
 8. An image processing method for processing input imagedata having a luminance component and other components, which aredefined in a prescribed color space, the method comprising: (a)processing first image data so as to emphasize the contrast of the firstimage data in accordance with a first emphasis amount, wherein the firstimage data correspond to the luminance component of the input imagedata; (b) processing second image data so as to emphasize the chroma ofthe second image data in accordance with the first emphasis amount,wherein the second image data correspond to said other components of theinput image data; and (c) generating new image data by combining thecontrast-emphasized first image data and the chroma-emphasized secondimage data.
 9. A computer-readable medium storing a program for causingthe computer to process input image data having a luminance componentand other components, which are defined in a prescribed color space, inaccordance with operations comprising: (a) processing first image dataso as to emphasize the contrast of the first image data in accordancewith a first emphasis amount, wherein the first image data correspond tothe luminance component of the input image data; (b) processing secondimage data so as to emphasize the chroma of the second image data inaccordance with the first emphasis amount, wherein the second image datacorrespond to said other components of the input image data; and (c)generating new image data by combining the contrast-emphasized firstimage data and the chroma-emphasized second image data.